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ABSTRACT 

The  reduction  in  false  alarm  probability  P(FA)  as  the 
number  of  displayed  echo  cycles  n is  increased  from  6 to  12  is 
investigated.  According  to  a predetermined  experimental  curve, 
the  value  of  S/N  is  reduced  with  increasing  n in  such  a manner 
as  to  maintain  a constant  detection  probability,  P(C).  Decreases 
in  P(FA)  as  great  as  0.19  were  observed.  To  obtain  a minimum 
P(FA)  while  retaining  constant  P(C),  a greater  number  of  echo 
cycles  are  required  for  the  initially  smaller  values  of  S/N 
for  n 
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I.  INTRODUCTION 

Investigations  have  been  made  concerning  the  expected 
decrease  in  false  alarm  probability,  P(FA),  as  the  number,  n,  of 
echo-cycles  is  increased  from  six  to  twelve,  one  echo-cycle  at  a 
time  being  added  to  an  intensity  modulated  display.  The  signal- 
to-noise  ratio,  s/N,  was  decreased  with  the  addition  of  each 
echo-cycle  according  to  a predetermined  experimental  curve^  in 
order  to  maintain  a constant  probability  of  detection,  P(C). 

The  decrease  in  P(FA)  was  0.19  for  a test  condition  pro- 
viding a P(C)  of  0.81,  but  remained  constant  after  the  number  of 
echo-cycles  had  been  increased  to  nine.  For  other  test  conditions 
a more  gradual  decrease  in  P(FA)  was  noted  as  n was  increased  to 
eleven. 

Plots  are  presented  of  P(C),  P(FA) , and  the  detectability 
index,  d',  as  a function  of  the  number  of  echo-cycles.  Receiver 
operating  characteristic  (ROC)  curves  were  obtained  by  requiring 
the  observers  to  use  a rating  scale  to  indicate  their  confidence 
in  each  decision. 

Section  II  describes  the  test  material  presented  to 
observers  and  a brief  discussion  of  the  test  procedures  employed. 
Results  and  their  interpretation  are  given  in  Section  III;  con- 
clusions are  listed  in  Section  IV.  The  appendix  describes  the 
multiple-alternative  decision  matrix  used  for  these  tests  and  its 
reduction  to  the  detectability  index,  d'. 
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II.  TEST  MATERIAL  AND  PROCEDURE 

The  film-strips  used  for  this  study  were  photographs  of 
a computer-generated,  intensity-modulated  display  with  intensity 
proportional  to  input  sample  amplitude.  Information  presented 
on  the  display  cathode-ray  tube  simulated  the  output  of  a single 
beam  of  the  AN/SQS-26  sonar  system  as  it  would  appear  on  the 
A-scan  indicator.  Generation  of  the  film-strips  has  been  des- 

p 

cribed  in  an  earlier  report. 

Three  groups  of  seven  film-strips  were  used,  each  group 
containing  successively  from  six  to  twelve  echo-cycles.  A film- 
strip consisted  of  150  frames,  each  with  common  values  of  n and 
s/N,  and  with  randomly  injected  signals  in  approximately  50  per- 
cent of  the  frames.  Also,  a signal  could  occur  in  one  of  six 
randomly  selected  positions  within  a given  frame.  The  values  of 
s/N  for  a given  number  of  echo-cycles  are  listed  in  Table  I for 
the  three  groups  of  film-strips.  The  values  of  s/N  for  Groups 
II  and  III  were  successively  reduced  from  corresponding  values 
in  Group  I because  of  the  high  detection  probability  afforded  by 
the  latter. 

The  film-strips  were  shown  to  a group  of  four  observers. 
These  observers  were  all  young  male  college  students,  three  of 
whom  had  recently  completed  a training  program;  the  fourth  has 
been  an  observer  for  over  a year.  During  a session,  the  ob- 
servers marked  their  answer  sheets  with  a 1,  2,  . . . , 6 if  they 
detected  a signal  in  one  of  the  six  positions;  otherwise,  a zero 
was  used . 

Results  were  classified  into  five  parts: 

(1)  P(C)  = Probability  of  correct  detection  (signal  identified 
and  located  correctly). 

^James  M.  Young,  "Marking  Density  Studies  for  the  AN/SQS-26  Sonar 
Equipment  A-Scan  Display,"  TRACOR,  Inc.,  Austin,  Texas,  TRACOR 
Document  No.  66-3I6-U,  Contract  NObsr-95149,  May  20,  1966. 
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TABLE  I 

VALUES  OF  S/N  FOR  THE  THREE 
GROUPS  OF  FILM-STRIPS 
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(2)  P(IT)  = Probability  of  incorrect  target  (signal  present, 

but  assigned  wrong  location) . 

(3)  P(M)  = Probability  of  a miss  (signal  present  but  not 

detec  ted) . 

(4)  P(FA)  = Probability  of  false  alarm  (noise  believed  to  be 

a s igna 1 ) . 

(5)  P(CR)  = Probability  of  a correct  rejection  (noise  believed 

to  be  noise) . 

P(C)  + P(IT)  + P(M)  = 1.0 
P(FA)  + P(CR)  -1.0 

The  time  allowed  for  viewing  the  film-strips  was  6 sec  ‘ 

per  frame  with  3 sec  between  frames  to  allow  the  observers  to 
mark  their  answer  sheets.  Each  film-strip  was  shown  four  or  more 

times  during  the  study.  ' 

A rating  scale  was  used  to  obtain  data  for  Receiver 
Operating  Characteristic  (ROC)  curves.  Using  a rating  scale, 
observers  were  made  to  adopt  more  than  one  criterion.  Not  only 
were  the  observers  required  to  note  signal  position,  but  they 
also  rated  their  responses  1 through  4,  depending  on  their  con- 
fidence that  a signal  was  present.  A zero  was  used  to  indicate  j 

that  they  were  almost  certain  no  signal  was  present.  Probabili- 
ties of  detection  P(D)  and  false  alarm  P(FA),  employing  the 
rating  scale  method,  are  defined  as 

4 4 

^ [Pk(C)  + Pk(IT)]  and  Pk(FA),  k-1,  ...,  4, 
k=l  k=l 

respectively.  The  ROC  curves  are  plots  of  P(D)  versus  P(FA). 

* ' 
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III.  RESULTS 

The  observed  results  for  all  of  the  film-strips  used  in 
this  study  are  presented  in  Table  II.  Discussions  of  the  results 
for  each  group  of  film-strips  follow. 

A . Group  I 

Figure  1 is  a plot  of  the  average  observer  perfor- 
mance, P(C)  and  P(FA),  on  each  film-strip  of  Group  I.  The  aver- 
age value  of  P(C)  throughout  this  group  is  0.81,  with  maximum 
and  minimum  values  of  0.82  and  O.78,  respectively.  For  n = 6, 
P(FA)  is  O.38,  but  decreases  to  0.25  for  n = J.  Although  a 
continuous  reduction  of  P(FA)  occurs  as  echo-cycle  histories  are 
successively  added,  a change  of  only  0.03  is  noted  from  n = 8 to 
n = 12.  Because  of  the  relatively  large  values  of  s/N  used  in 
Group  I,  and  the  consequent  ease  in  discerning  signals,  detect- 
ability is  not  enhanced  for  9 < n s 12,  as  depicted  by  the  d' 
curve  in  Fig.  1.  Thus,  for  the  number  of  echo-cycles  studied,  a 
lower  limit  to  P(FA)  was  approached  for  n = 9,  with  almost  no 
further  decrease  through  n = 12. 

Figure  2 shows  the  ROC  curves  for  Group  I,  obtained 
by  the  rating  scale  method.  The  twelve  echo-cycle  film-strip 
was  not  used.  The  ROC  curves  for  all  other  Group  I film-strips, 
with  the  exception  of  the  one  containing  11  echo-cycles,  show 
little  spread. 

Since  P(FA)  reaches  a minimum  so  rapidly  with  increas- 
ing n,  and  because  of  the  high  probability  of  detection  for  Group 
I,  two  other  groups  of  film-strips  were  generated,  using  smaller 
values  of  s/N  for  a given  n. 

B.  Group  II 

The  second  group  of  film-strips  was  generated  using 
a smaller  value  of  S/N  for  a given  number  of  echo-cycles  than  in 
Group  I (See  Table  I).  Averages  of  P(C)  and  P(FA)  for  Group  II 
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FIG.2 -OBSERVER'S  ROC  CURVES  FOR  GROUP  I FILM  STRIPS. 

THE  INDICATED  S/N  ARE  AT  THE  INPUT  TO  THE  DISPLAY. 
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are  plotted  in  Fig.  3»  and  ROC  curves  for  this  group  are  shown 
in  Fig.  4.  The  average  P(C)  for  the  entire  group  is  0.66;  the 
decreasing  trend  in  P(FA)  extends  to  eleven  echo-cycles.  Total 
decrease  in  P(FA)  , from  six  to  eleven  echo-cycles,  is  0.16. 
Surprisingly,  each  P(FA)  for  the  film-strips  in  Group  II  is  less 
than  the  corresponding  P(FA)  for  Group  I.  The  detectability 
index,  d',  for  each  Group  II  film-strip  is  less  than  the  d'  value 
for  the  corresponding  film-strip  of  Group  I. 

The  ROC  curves  for  Group  II  (Fig.  4)  relate  almost 
identically  to  the  corresponding  d'  values  listed. 

C . Group  III 

To  increase  the  P(FA)  for  n - 6,  from  that  obtained 
for  the  Group  II  film-strips,  a third  group  of  film-strips  was 
prepared  with  an  even  smaller  value  of  s/N.  The  average  P(C)  for 
this  group  was  0.54;  the  decrease  in  P(FA)  of  0.13  occurred  from 
n = 6 to  n = 12  (Fig.  5).  Correspondingly,  d’  increased  from 
0.85  to  1.34. 

Since  the  twelve  echo-cycle  film-strip  showed  a sub- 
stantial reduction  in  P(FA)  from  that  for  six  echo-cycles,  the 
value  of  S/N  for  Group  III  was  low  enough  to  prevent  P(FA)  from 
reaching  a lower  bound  for  n < 12.  The  decrease  in  P(FA)  with 
increasing  n was  more  gradual  for  these  smaller  values  of  S/N, 
but  continues  as  more  echo-cycles  are  added. 

Observer's  ROC  curves  for  Group  III  are  shown  in  Fig. 
6.  The  approximate  spread  of  the  ROC  curves  corresponds  to  the 
range  of  d'  values  listed  in  Table  II.  Grouping  of  the  curves  is 
not  in  the  expected  order,  probably  because  of  the  large  values 
of  P(IT)  and  P(FA).  Figure  6 thus  indicates  that  less  consistency 
is  afforded  by  Group  III,  in  agreement  with  the  P(C)  graph  of 
Fig.  5. 

The  average  P(C)  for  all  groups  was  O.67.  Observed 
values  of  the  detectability  index  d'  were  used  to  normalize  all 
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FIG. 4 -OBSERVER'S  ROC  CURVES  FOR  GROUP  H FILM  STRIPS. 

THE  INDICATED  S/N  ARE  AT  THE  INPUT  TO  THE  DISPLAY 
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values  of  P(FA)  to  a constant  P(C)  = 0.65.  This  normalized  value 
of  P(FA)  is  designated  hereafter  by  P*(FA).  (Note:  This  value 
rather  than  0 .67  was  chosen  merely  for  convenience.)  Figure  7 
is  a plot  of  P*(FA)  as  a function  of  the  number  of  echo-cycles, 
n,  in  each  group.  The  separation  of  Group  I and  Group  II  curves 
is  small;  Group  III  values  of  P*(FA)  are  substantially  higher 
than  those  for  either  of  the  first  two  groups.  A decrease  in 
P*(FA)  for  Group  III  of  0.15  occurs  as  the  number  of  echo-cycles 
is  increased  from  six  to  twelve.  The  reductions  in  P*(FA)  for 
Groups  I and  II  are  0.09  and  0.12,  respectively. 
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FIG.7- VALUES  OF  P*(FA)  FOR  A CONSTANT  PROBABILITY  OF  DETECTION  P'(C)  = 0.65 
FOR  THE  THREE  GROUPS  OF  FILM  STRIPS  LISTED  IN  TABLE  I. 
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VI.  CONCLUSIONS 

On  the  basis  of  the  experiments  reported  here  the  follow- 
ing conclusions  are  made: 

(1)  The  likelihood,  or  probability,  of  false  alarm 
P(FA)  decreases  as  the  number  of  echo-cycles  is  increased.  The 
reduction  occurs  even  while  the  input  value  of  S/N  is  reduced 
with  each  additional  echo-cycle,  the  reduction  in  S/N  being 
determined  by  previous  data  designed  to  hold  the  likelihood 

of  a hit  P(C)  constant. 

(2)  Reduction  in  P(FA)  is  curvilinearly  related  to  the 
initial  S/N.  A greater  number  of  echo-cycles  is  required  to 
attain  a constant  lower  bound  on  P(FA)  as  the  value  of  S/N  is 
reduced,  while  maintaining  a constant  P(C).  This  decrease  in 
P(FA)  becomes  more  gradual,  as  each  echo-cycle  is  added,  for 
smaller  values  of  S/N. 

(3)  P(FA)  approaches  a lower  bound  as  the  initial  S/N 
is  made  smaller.  In  order  for  P(FA)  to  continue  decreasing  as 

n is  increased,  the  value  of  S/N  must  be  large  enough  to  provide 
a constant  value  of  P(C).  As  the  value  of  S/N  is  decreased, 

P(IT)  increases  and  P(C)  begins  to  fluctuate. 
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APPENDIX 

I.  DETECTABILITY  INDEX 

A psychophysical  measure  d'  (detectability  index)  is 
used  in  this  study  as  an  indication  of  observer  performance.  The 
basis  for  this  measure  will  be  explained,  and  reinterpreted  in 
terms  of  the  stimulus-response  matrix  employed. 

Tanner  and  Birdsall^  have  defined  d1  for  the  binary 
detection  problem,  i.e.,  the  two-alternative,  forced-choice 
situation.  As  used  in  this  memorandum,  d'  may  be  regarded  either 
as  a function  of  signal  effectiveness  or  as  a measure  of  observer 
sensitivity.  The  detectability  of  a signal,  as  defined  by  Tanner 
and  Birdsall,  is  the  magnitude  of  (2E/N0)^2  necessary  for  the 
performance  of  an  ideal  receiver  to  match  the  performance  of  the 
receiver  being  studied  (in  our  case,  a human  observer).  Here  E 
is  the  signal  energy  and  NQ  is  the  noise  power  per  unit  bandwidth. 

The  detectability  index  d'  may  be  obtained  from  two 
normal  probability  density  distributions,  one  for  noise  alone, 
the  other  for  signal-plus-noise.  Both  distributions  are  assumed 
to  have  equal  variance  (Fig.  A-l).  Separation  of  the  means  of 
the  overlapping  distributions,  measured  in  units  of  standard 
deviation,  o,  is  the  detectability  index  d'. 

An  idealized  ROC  curve  may  be  constructed  from  the  normal 
distributions  (Fig.  A-l)  by  requiring  the  observers  to  vary  their 
criterion,  C (e.g.,  by  use  of  a rating  scale  as  discussed  in 
Section  III).  The  area  under  the  S+N  distribution  to  the  right 
of  C is  P(C),  and  P(FA)  is  the  area  under  the  N distribution  to 
the  right  of  C.  Each  pair  of  areas,  for  a given  C,  determine 

a point  on  an  ROC  curve.  In  this  instance  (normal  distributions 

/ 


^W.  P.  Tanner,  Jr.,  and  T.  G.  Birdsall,  "Definitions  of  d'  and  n 
as  Psychophysical  Measures,"  J.  Acoust.  Soc.  Am.,  20,,  922-928, 
(1958) . 
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of  equal  variance),  the  ROC  curve  is  symmetric  about  the  nega- 
tive diagonal,  and  d ' is  a measure  of  the  distance  from  the 
chance  line  to  the  intersection  of  the  curve  with  the  negative 

diagonal.  Allowance  for  test  hypotheses  with  unequal  variance 

4 5 

is  discussed  by  Clarke,  Birdsall,  and  Tanner  . Jeffress  has 
also  considered  the  effects  of  unequal  variances,  as  well  as 
those  due  to  skewness  of  the  N and  S+N  distributions. 

The  foregoing  has  assumed  a yes-no  type  of  experiment^, 
having  the  four-element  stimulus-response  matrix  shown  below. 

STIMULUS 
S+N  N 
H FA 
M CR 

The  particular  type  of  experiment  with  which  this  note 
is  concerned  involves  a multiple-alternative  decision  problem. 
Discrimination  must  be  made  among  more  than  two  hypotheses,  e.g., 
a signal,  if  it  occurs,  may  be  in  any  one  of  six  positions.  The 
resulting  stimulus-response  matrix: 


STIMULUS 


w 

C/3 

2 

S+N 

C 

N 

C 

IT 

- Hit 

- Incorrect  Target 

O 

P* 

C/3 

S+N 

IT 

FA 

M 

FA 

- Miss 

- False  Alarm 

UJ 

od 

N 

M 

CR 

CR 

- Correct  Rejection 

w 

C/3 

§ 

tu 

C/3 

W 

cd 


S+N 

N 


R.  Clarke,  T.  G.  Birdsall,  and  W.  P.  Tanner,  Jr.,  "Two  Types 
of  ROC  Curves  and  Definitions  of  Parameters,"  J.  Acoust.  Soc . Am., 
31,  629-630,  (1959). 

5Jeffress,  Lloyd  A.,  "Stimulus-Oriented  Approach  to  Detection," 

J.  Acoust.  Soc.  Am.,  2§.»  7^6-77^?  (1964). 

^An  excellent  treatment  of  this  subject  is  given  in:  D.  M.  Green 
and  J.  A.  Swets,  Signal  Detection  Theory  and  Psychophysics,  John 
Wiley  and  Sons,  Inc.,  New  York,  (19bb). See  especially  the 
material  in  Chap.  2. 
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contains  five  elements  rather  than  four.  The  extra  element  for 
the  multiple-alternative  situation  is  IT  (incorrect  target), 
meaning  a signal  actually  occurred  but  its  location  was  incor- 
rectly noted  by  the  observer.  Now,  possible  interpretations  of 
the  IT  response  in  terms  of  the  four-element  matrix  are:  (1) 

Miss,  (2)  False  Alarm,  or  (3)  Hit,  depending  upon  the  relative 
cost  factors  of  each  element.  In  our  case,  no  relevant  costs 
may  be  assigned. 

Middleton' , in  his  formulation  of  the  multiple-alternative 
decision  situation,  includes  the  element  IT  with  the  element  M 
in  his  determination  and  minimization  of  average  risk.  A conse- 
quence of  this  choice  is  that  the  point  on  an  ROC  curve  corres- 
ponding to  P(FA)  = 1.0  will  have  P(D)  < 1.0. 


